The present invention is directed generally to apparatus and methods for simulating various gestures of a pointing device with multiple switches, such as a two button mouse in a computer system, and for providing feedback thereto, and more specifically to simulating gestures such as the left click of a mouse, the right click of a mouse, and mouse dragging by manipulation of a stylus in conjunction with a touch-sensitive computer display, as well as generating appropriate visual or other feedback in response to certain gestures.
Typical computer systems, especially computer systems using graphical user interface (GUI) systems such as Microsoft WINDOWS, are optimized for accepting user input from two discrete input devices: a keyboard for entering text, and a pointing device such as a mouse with two or more buttons or other selectors for driving the user interface. Virtually all software applications designed to run on Microsoft WINDOWS are optimized to accept user input in the same manner. For instance, many applications make extensive use of a primary switch of the pointing device such as the left button of a mouse (a gesture commonly referred to as a xe2x80x9cleft clickxe2x80x9d of a mouse) to select an object, and a secondary switch such as the right button of the mouse (a gesture commonly referred to as a xe2x80x9cright clickxe2x80x9d of a mouse) to display context-sensitive command menus. It is noted that other operating systems incorporate left and right click operability as well. The user may generate other gestures using the mouse such as by clicking the left or right button of the mouse and moving the mouse while the button is depressed (a gesture commonly referred to as either a xe2x80x9cleft dragxe2x80x9d or a xe2x80x9cright drag,xe2x80x9d respectively).
While such mouse gestures are of course easily done with an actual mouse, it is not always convenient for a user to use a mouse with a computer even though the computer may have a GUI system. For instance, when using a portable system there may not always be a surface available upon which to place a mouse. Some systems utilize a touch-sensitive display screen with a stylus as an input device. The user places the stylus tip upon the touch-sensitive display screen at various locations to control objects displayed on the display screen. A problem with this type of system is that the stylus is limited in the types of actions that may be easily performed as compared with a mouse. For instance, the typical stylus can perform only three types of movements: placing the stylus tip onto the screen, moving the stylus tip across the screen, and removing the stylus tip from the screen. While a user may operate a remote toolbar to select a different tool, the excessive movement renders the control of the system laborious, inconvenient, and slow. Using the tip of the stylus-type device, there is presently no convenient way to simulate or distinguish between, for example, a right click as opposed to a left click as interpreted by the operating system.
In order to make the full range of interface features accessible to users of such a stylus-based computer, there is a need for an intuitive way of simulating mouse gestures with the stylus. Although not necessary, it is preferable and more useful that any new way of simulating mouse gestures be compatible with existing software applications that conventionally are used with a mouse.
There is also a need for providing helpful feedback to the user of the stylus to indicate whether gestures made with the stylus are those that are intended by the user. This is useful with a stylus-type input device since there is conventionally little to no natural tactile or auditory feedback when using a stylus than there is when using a mouse (e.g., the buttons of a mouse often have a definitive mechanical xe2x80x9cclickxe2x80x9d feel and sound when pressed). As will be seen, such feedback is even more useful when provided in conjunction with various gesture simulation techniques of the present invention.
As discussed in the various copending patent applications incorporated herein by reference, aspects of the present invention are directed to a tablet-like computer that may be used for directly writing on a touch-sensitive display surface using a stylus. The computer may allow the user to write and to edit, manipulate, and create objects through the use of the stylus. Many of the features discussed in these copending applications are even more useful in combination with various aspects of the present invention discussed herein.
An aspect of the present invention is directed to methods and apparatus for simulating gestures of a pointing device, such as a mouse, trackball, or joystick, by use of a stylus on a touch-sensitive display surface. For example, a left click of a mouse may be simulated, e.g., where the user holds the stylus down on the touch-sensitive display surface without substantial movement and then removes the stylus from the display surface before the expiration of a threshold amount of time. A right click of a mouse may be simulated, e.g., where the user waits until at least the threshold amount of time before removing the stylus. Where the stylus is moved along the display surface during certain stylus gestures, left or right dragging may thereby be simulated.
Another aspect of the present invention is directed to methods and apparatus for providing feedback to a user in a stylus-based touch-sensitive display surface computer system. The user may accordingly be given some indication that the user has performed a particular stylus gesture, e.g., by displaying a visual object on the display surface to indicate a particular status. Such feedback may be responsive to certain stylus gestures such as pressing and holding the stylus against the display surface for at least a certain minimum amount of time. More generally, the feedback may be in the form of a xe2x80x9cstate change indicator,xe2x80x9d which may be visual on the display surface such as an icon, bitmap, or other visual object or indicator, and/or auditory such as a sound emitted from the computer""s speakers. The state change indicator may be animated and or otherwise vary with stylus hold time. Significant advantages of this aspect of the invention are realized in conjunction with the various features discussed in the copending patent applications incorporated herein by reference.
These and other features of the invention will be apparent upon consideration of the following detailed description of preferred embodiments. Although the invention has been defined using the appended claims, these claims are exemplary in that the invention is intended to include the elements and steps described herein in any combination or subcombination. Accordingly, there are any number of alternative combinations for defining the invention, which incorporate one or more elements from the specification, including the description, claims, and drawings, in various combinations or subcombinations. It will be apparent to those skilled in the relevant technology, in light of the present specification, that alternate combinations of aspects of the invention, either alone or in combination with one or more elements or steps defined herein, may be utilized as modifications or alterations of the invention or as part of the invention. It is intended that the written description of the invention contained herein covers all such modifications and alterations.